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ADAMTV^ EVALUATION METHOD 
AND AD APTIVE EVALUATION APPARATUS 

5 FIELD G J THE INVENTION 

The present invention relates generally to an evaluation method and an evaluation 
apparatus. More particularly, the present invention relates to an adaptive evaluation method 
and an adaptive evaluation apparatus. 

10 BACKGROUND INFORMATION 

A common prior art testing me&od is illustrated, for example, in Figure 1 . The 
testing method shown in Figure 1 may be computer-administered. In the method illustrated 
in Figure 1, the method is started m step SlOO, duriiig which initialization and start-up 
sequences may be performed. The step SlOO rnay also include steps of gathering and 

15 recording such information as tiie test-taker's name, address aiid other identifying 

information. In addition, the step SlOO may mclude a step of selecting a particular one of 
several available tests stored in, for example, a computer memoiy device, such as a 
CD-ROM, a DVD-ROM, a magnetic medium or the like. 

Thereafter, in step 8102, the value of the variable Score is initialized so that 

20 Score = 0, and in step S104, the value of the question counter / is initialized so that i = 1. In 
the step SI 06, the question Q(i) corresponding to the valiie of the question counter i is 
presented to the test-taker. Hie questions may be presented to ftie test-taker by, for example, 
displaying the questions on a displaying device. The questions may be in the form of, for 
example^ true-false questions,, multiple, choice questions or any other of the myriad of 

25 question types known in the art or any combination of the foregoing. The questions may be 
stored in a computer database, CD-ROM, DVD-ROM or the like and may be presented in a 
predetermined sequence or in a random sequence. The question may be stored locally or may 
be remotely stored and accessed, for example, via the Intemet 

Thereafter, in step SI 08, the test-taker enters his answer ^(0 corresponding to the 

30 question Q(i) presented in step S106. The test-taker may enter his answer by, for example, 
keyboard entry, mouse entry, toUch-screen entry, voice recognition data acquisition or the 
like. Thereafter, in step SI 10 it is determined whether the answer A({) is correct by 
comparing the answer ^(0 entered by the tiest-taker with the known correct answer to 
question Q(i). If, in step SI 10 it is deterrnineid that the answer ^(0 is correct, the value of the 
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variable Score is incremented in step SI 12 by the value v(i), which may be dififerent for 
different questions based on, for exaniple, the Qpe and/or relative difficulty of the question. 

It is then determiiied in step S114 whetiiier the Valtie of the question counter i is equal 
to the value of ^, the predetenfiined number of questions to be administered in the test. If the 
value of / is determmed to not be equal to A, indicating that ti[ie test has not been completed, 
the value of the question counter / is incremented in step SI 16, and the method returns to step 
8106, where flie next question 6(0 is presented. Hife stq)s 8^106, S108, SI 10, SI 12, S*l 14, 
SI 16 are repeated until it is detennined in step SI 14 that the value of tiie question counter / is 
equal to flie value of thereby indicating that the test has been completed. 

Thereaft^, the step S120 is performed. The step S120 may include the steps of score 
recording and reporting and appropriate termmation and shut-down procedures. 

The foregoing prior art testing method has been heretofore applied to individually 
administered tests and group administered tests. In an individuaUy administered test, the test- 
taker is presented with a series of standardized questions. The administrating entity scores 
the test based on the answers entered by the test-taker. The scoring iriay be calculated and 
reported based on sub-categories as defined in the test structure or may simply be a scalar 
value representing the test-taker' s score. 

In a group administered test, the test-taker's score is calculated-based on the number • 
of correct answers. The number of incorrect answers tiiay also be a component of the score. 
The score is calculated using various algorithms known in the art. 

The aforementioned testing method is considered to be a linear method. That is, the 
questions are presented in order, either in a predefined sequence or a random sequence, until 
the predetermined number of questions have been answered. The method described above is 
rigid, in that it cannot accommodate various skill or aptitude levels within the test regimen, 
■oMier than reflecting these levels within the final score. The final score is conventionally 
represented as a smgle scalar number (z.e., a "total" score) or as a series of scalar numbere, 
each representing a "total*' score m a series of subject areas. This type of conventional 
scoring is merely an attempt to locate each individual at some ordinate. These scalar values, 
however, do not provide any significant information regarding the individual or that 
individual's skill level. It is, thefefore, an object of tiie present invention to provide an 
adaptive evaluation method and an adaptive evaluation apparatus. 
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SUMMARY 

The above and other beneficial objects of the present invention are most effectively 
attained by providing an evaluation method and an evaluation apparatus as described and 
claimed herein. In one embodiment, a plurality of questions are represented in a matrix of N 
items m each of a number of M tracks. The questions aire presented in order of increasing 
difBculty fix>m a fLat randomly selected track. After a question answered incorrectly, the 
track is changed aktd the next question presented is oflesser difficulty than the question 
answered incorrectly. After a predeitermined iiimiber of track changes, the evaluation is 
considered completed after a predetermined number of questions have been answered 
incorrectly. The method may be embodied in a computer readable medium or may be 
embodied in a'computer system. 

BRIEF DESCRIPTION OF THE DRAWINGS 
In the accompanying drawings: 
Figure 1 is a flowchart of a prior art testing method; 

Figure 2 is a schematic view of a memory architecture storing a plurality of evaluation 
questions used in an evaluation method and an evaluation apparatus according to the present 
invention; ... 

Figure 3 is a schematic view of an embodiment memory architecture of the present 

invention; 

Figure 4 is a flowchart of the evaluation method of the present invention; 

Figure 5 is a further flowchart of the evaluation method of the present invention; and 

Figure 6 is yet anolher flowchart of the evaluation method of the present invention. 

DETATT.yn DESCRIPTION 

Those skilled in the art will gain an appreciation of the present invention when viewed 
m conjunction with the accompanying drawings of Figures 1 through 6, inclusive. The 
individual reference characters designate the same or shnilar elements fluroughout the several 
views. 

Referring now to Figure 2, there is seen a schematic view of a memory architecture 
(M, N) 10 storing a plurality of questions to be administered by the evaluation method and 
the evaluation apparatus according to the present invention. It will be appreciated fliat the 
memory architecture (M, N) 10 may be in the form of, for example, a CD-ROM, a 
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DVD-ROM, an optical medium, a magnetic medium, a RAM, a ROM or any other type of 
storage medium. It will also be appreciated that the memory architecture (M, N) 10 may store 
the plurality of questions locally, permanently, temporarily and/or removably, on the 
evaluated person's computing device or may store the plurality of questions remotely so that 
the evaluated person accesses the memory architecture (M, N) 10 by, for example, Internet 
commimication or other network or remote communication. 

The imem^ory architecture (M, N) 10 illustrated in Figure 2 is represented 
schematically, for clarity and understanding, as a two-dimensional array or matrix, although 
the memory architecture (M, N) 10 is in no way limited to the schematic representation 
shown. The array shown in Figure 2 mcludes M rows of tracks T, . . . and N columns of 
items /j . . . 4: Thus, the memory architecture (M, >J) 10 is an M x N matrix representing 
questions Q^ , . . . or 2(1, 1) . . . Q(M, N). The foftnats Q^^ and Q(M,N)w^ used 
herein mterchangeably. The questions Q^^, . . may be, for exarhple, multiple choice 
questions, true-false questions or any other of flie myriad types of questions or any 
combination thereof. 

Referring now to Figure 3, there is seen an embodiment memory architecture (M, N) 
1 0, which includes four tracks T, ... 3; and ei^t items /i . . . /g, defining a 4 x 8 matrix. It 
should be understood that the embodiment memory architecture (M, N) 10 illustrated in 
Figure 3 is merely exemplary and that memory architecture (M, N) 10 may include any 
number of tracks and any number of items. 

Referring now to Figure 4, there is seen a flowchart of the evaluation method of the 
present invention. The method is started in step S200, and initialization is performed in step 
S202. The initialization step S202 may include such steps as acquiring and storing 
identifying information from the evaluated person, selecting one of a plurality of available 
evaluation subjects or regimens, variable mitializaiion and the like. 

Thereafter, in step S204, the variable TotalScore is initialized so that TotalScore = 0. 
Similarly, in step.S206, the elements of the one-diniiensional. array TracMm) are initialized so 
that TracKm) = FALSE for aU valid values of w, thait is, 1 to M^^. Each element of the array 
TracMpi) stores a boolean value of TRUE (1) or FALSE (0) to indicate whether the track m 
has been used. In step S208, the variable TrackSwitches is initialized so that 
TrackSwitches = 0. The variable TrackSwitches indicates the number of times tiie track has 
been changed or switched, as described hereinbelow. the question counter i is initialized in 
step S210 so that / = 0, and the item counter iV is initialized in step S212 so thatiS^= 1. 
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After these variable initialization steps, one of the tracks is randomly selected in step 
S214 by assigning to the variable M so that M= END, wherein END represents a random 
integer between 1 andil4^. In step S216, the element Track(M) is assigned 
Track(M) = TRUE, which will later be used to prevent the track M from being used 
subsequently. In step S2 18, the question e(A/,jy) is presented to the evaluated person. The 
question Q(M, N) may be presented by, for example, displaying the question Q(M, AO on a 
displaying device of the evaluated person's computing device. Thereafter, in step S220, the 
evaluated person's answer ^(Af, N) to question Q(M, N) is acquired by, for example, 
keyboard entry, mouse eiitry, touch-scareen entry, voice-recognition data acquisition or the 
like. Thereafter, in step S222, it is determined whether the answer ^(M, N) acquired in step 
S220 to question Q(M, N) presented in step S218 is correct If it is determined in step S222 
that the answer ^(Af, iV) to question Q(M, N) is correct, the method proceeds to step S224, 
which is more fiiUy described in Figure 5. If it is determined in step S222 that tiie answer 
AiM, N) to question Q(M, N) is incorrect, the method proceeds to step S226, which is more 
fully described in Figure 6. After either step S224 Or S226 is performed, step S228 is 
performed, in which the question couniter i is incremented so that i = i + 1. Thereafter, in step 
S230 it is determined whether the value of iV^ is equal to the value of r, which represents the 
total number of items. If die value ofNis determined to be equal to the value of rin step 
S230, the step S232 is perfonned, which terminates the method and performs any necessary 
shut-down, data recording and score reporting operations or the like. If the value of Nis 
determined to not be equal to the value of Jin step S230, the method returns to step S218 to 
present the next question Q(M, N). 

Referring now to Figure 5, there is seen a flowchart of the method step S224, which is 
performed after it is determined in step S222 that the answer ^(Af, N) to question Q(M, N) is 
correct. In step S234 the value of ^core(0 is assigned so that Score(i) = v(Af, N), wherein 
v(Af, N) represents the value of the correct answer to the question Q(M, N). The value of 
v(M, N) may be based on, for example, the type and/or relative difficulty of the question 
Q(M, iV). It will be appreciated that aU questions Q(M, N) for any given AT are preferably of 
substantially equal diflSculty. That is, all questions Q(M, N) m any given column of the 
memory architecture 10 illustrated in Figures 1 and 2 are of substantiaUy equal difiiculty. 
Thereafter, in step S236, the value of tiie boolean variable CorrecHi) is assigned so that 
CorrecKi) = TRUE, which will later be used to determine whethereach of a sequence of 
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questions were answered correctly or incorrectly. Thereafter, in step S238, the value of 
TotalScore is assigned so fhat TotalScore = TotalScore + Score{i). 

Thereafter, in step S240, it is determined whether k number of questions have been 
answered correctly. Step S240 may be performed using an AND operation, such as: 

Correct(i) AND Correct{i - 1) AND . . . Correct{i -k+l) (1) 

If it is determined in step S240 that k number of questions hkve been answered correctly, that 
is, if the expression (1) above is evaluated as TRUE, the value of iVis iiicremented by 
assigning to AT so that N==N+Km step S242, flieJeby jumping forward number of 
questions. If it is determined in step S240 that k number of questions have not been answered 
correctly, thenthe value of AT is incremented by assigning to i\^so that N+ 1 in step S246. 
After either of step S242 or step S246 is performed, step S244 is performed in which the 
value of question counter / is incremented so ttiat / = r+ 1. Thereafter, control is returned in 
step S248. 

Referring now to Figure 6, there is seen a flowchart of the method step S226, which is 
performed after it is determined in step S222 fliat the answer N) to question QiM, N) is 
incorrect. In step S250, the value of Scoreii) is assigned so that Score(i) = 0. Thereafter, the 
value of Correctii) is assigned in step S252 so that Correct(i) = FALSE. Thereafter, in step 
S254, it is determined whether the number of track changes as represented by TrackChanges 
is less than a predetermined value L. If the number of track changes TrackChanges is 
determmed to be less than the value of Z, a new track M is randomly selected in step 8256. In 
step S258, it is determined whether the newly selected track A/has been previously used by 
determining whether the value of Track{M) is TRUE. If it is determined in step S258 that the 
newly selected track Mhas previously been used, step S256 is performed again to randomly 
jelect yet another new track M The steps S256 and S258 are repeated until the newly 
selected track M, randomly selected in step S256, has not been previously used. Thereafter, 
in step S260, the value of the item cotmter iSTis decremented by tiie value of iVg, thereby 
movmg back in the question sequence in the newly selected track. In step 8262, it is 
determined whether the item counter iS^ as decrtimented in steps S260 is less than 1 and, if so, 
the value of iV is assigned so that 1, which is the lowest vaUd value ofN. Thereafter, in 
step S266, the value of TrackSwitches is incremerited so that 

Tracl^witches = TrackSwitches + 1. Control is then returned in step S270 to the main 
routine shown in Figure 4. 
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If it is determined in step S254 that the number of track switches TrdckSwitches is not 
less than the value of thereby indicating that the maximuni permissible number of track 
changes has been performed, it is determined in stejp S268 whether S number of questions 
have been answered incorrectly in sequence. The determination performed in step S268 may 
be performed by evaluating the e^tpression: 

Correct(fj AND Correct(i - 1) . . .Correci{i - S + 1) Cor:':2t{i) (2) 

If the expression (2) is evaluated to be FALSE, it is determined in step S268 that S number of 
questions have been answered incorrectly in sequence. Then step S272 is performed, in 
which the appropriate termination steps are perfonned, such as those performed in step S232, 
described hereinabove. If it is determined in step S268 that S nxmiber of questions have not 
been answered incorrectly in sequence, Haesa control is returned in step S270 to the main 
routine shown in Figure 4. 

The foregoing method will be fur&er desiaribed with reference to flie exemplary 
embodiment illustrated in Figure 3. As described above, the memory architecture (M, N) 10 
shown in Figure 3 consists of four tracks T, ... 74 and eight items . , . /g in each of the 
tracks . . . T^, The method is started in step S200 as described above, and the steps S202, 
S204, S206, S208, S210 and S212 are thereafter performed as also described above. In stqp 
S206, the values of each element TracHpi) is initialized, for m == 1 to 4. In step S214, the 
value of Mis randomly assigned. As can be seen in Figure 3, the value of Mis randomly 
assigned the value of 2, corresponding to the second track T^. Thereafter, in step S216 the 
value of the element Trackil) is assigned so that Track{2) = TRUE, thereby indicating that 
the second track will not be available subsequently for a track change. In step S218, the 
question Q(2, 1) is presented, and, thereafter in step S220, the answer ^(2, 1) is acquired. In 
the example illustrated in Figure 3, the question 2(2, 1) is correctly answered, and, therefore, 
in step S222 it is determined that ^(2, 1) is correct, thereby performing the step S224, In step 
S234, the value otScoreiX) is assigned so ihatScore(X) = v(2, 1). . For simplicity, the value of 
v(A<f, N) will be 1 for all questions. Thus, in step S234, the value of 5bore(l) is assigned so 
that&ore(l)=l. 

Thereafter, m step S236, the value of the element Correct(X) is assigned so that 
CorrectiXi =.TRIJE, and in step S238, the value of TotalScore in incremented by the value of 
Score(l). At this stage, the value of -Tbto/Sfcore^l. In the example illustrated in Figure 3, 
tiie value of* is assigned so ttiat Since only dlie questioii has been presented, the 
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determination of step S240 returns a FALSE value, thereby causing step S246 to be 
performed to increment A'^ so that iV = 2. Returning in step S248 to the main routine shown in 
Figure 4, the step S228 is performed, in which the question counter i is incremented so that 
/ = 1 to indicate that one question has been ans\^'ered. It is determined in step S230 that the 
5 total number of questions Thave not been answered, thereby returning to step S2 1 8 to present 
the next question g(2, 2). 

In the example illustrated in Figure 3, the question Q(2, 2) is also answered correctly. 
The method then progresses as described above with respect to question g(2, 1). The next 
question g(2, 3) is presented and also answered correctly. Since three questions have been 
0 answered correctly in sequence, the determination made in step S240 returns TRUE. That is 
Correcti^) AND Correct{2) AND Correct{l) = TRUE, thereby causing step S242 to be 
performed. In the exemplaiy embodiment illustrated in Figure 3, the value of 2, thereby 
assigning to so that //= 5, effectively skipping question 0(2, 4). The method progresses to 
present question Q{2, 5), which is answered correctly, and then to present question 0(2, 6), 
5 which is answered incorrectly. 

Because question Q(2, 6) is answered incorrectly, the method proceeds to step S226. 
The value of Correct{5) is assigned so that Correct(5) = FALSE, indicating that the fifih 
question, which is question 2(2, 6), was answered incorrectly. In the exranplaiy 
embodiment, L, the maximxim number of track switches, is assigned so that Z = 2. Since 
0 TrackSwitches = 0 at this stage, the metiiod proceeds to step S256, in which the value of Af is 
randomly selected so that = 4, indicating that the next track will be track T^, In the 
exemplary embodiment illustrated in Figure 3, the value of iV^ is assigned so that = 3, 
indicating that after a track change, the next question presented will be three before the most 
recent question answered incorrectly. Thus, at this stage, after answering question 2(2, 6) 
5 mcorrectly, 4 and i\r= 3. In step S266, the vai je of TrackSwitches is incremented so that 
TrackSwitches = 1, indicative of the fact that the first track switch has occurred. Returning in 
step S270 to the main routme shown in Figure 4, the next question Q(4, 3) is presented. 
Following the progression of questions shown in Figure 3, questions 2(4, 3) and 
Q(4, 4) are presented and answered correctly. However, question Q(4, 5) is answered 
0 incorrectly, thereby invoking another track change. The second track change causes track 
to be used, begiming with question 2(1, 2), thfee questions back fix)m the 1^ 
answered question 2(4, 5). The process continues, presenting-questions 2(1, 2), 2(1, 3) and 
2(1, 4), which are all answered correctly. Since three questions were answered sequentially. 
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the question 5) is skipped and question 6) next presented Question Q(l, 6) is 
answered incorrectly. In the exemplary embodiment illustrated in Figure 3, the maximum 
track changes permitted, as represented by the value of Z, is two, which, as seen in Figure 3 
has, at this stage, occurred, no further track changes are performed. In the exemplary 
embodiment Dlustrated in Figiare 3, the value of S is assigaed to that »S = 3. Since at this 
stage, three questions have not been answered incorrectly in sequence, the next question 
2(1, 7) is presented The question 7) is also answered incorrectly, causing the next 
question 2(1, 8) to be presented Question g(l, 8) is also answered incorrectly. At this stage 
S number of questions, that is, three questions, have been answered incorrectly in sequence, 
thereby terminating the method by performing step S272. 

In a preferred embodiment of the present invention, each of the questions Q(M, N) is 
further defined by an evaluation structure, which includes a base or prime question gp(M, N) 
and a series of subcon:Q)onents Q^iiM, N) . - . QJMyN). Each of the subcomponents 
2si(Af, iV) . . . QJiM^ N) represents an alternate presentation, or "hint," of the respective prime 
question gp(Af, N). For example, a prime question may presented as: 

2 + 5==? 

where the evaluated person must insert fbo correct number at the In the prior art testing 

method described above, if the test-taker entered an incorrect number at the "?**, the answer 

would be considered **wrong" and the next question would be subsequently presented 

However, according to the present evaluation method, if an incorrect number was entered at 

the above, the following subcomponent would be presented: 

2 
-fS 
? 

The evaluated person would agam be required to faisert his answer at the If an incorrect 
answer were entered at the "?'^ a furthersubconq)dnent nMy be presente^^ In the-foregoing 
example, a further subcomponent may be, for example, a picture of seven balls, the evaluated 
person being asked to identify the total number of balls. 

It should be appreciated that by presenting a question based on a prime question and a 
series of subcomponents,- the zone of proximal development of the evaluated-person may be 
measured. Instead of presenting a scalar "score," as is usuaUy done in the prior art testmg 
method described above, the results presented in fhfe evaluation method according to the 
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present invention are in the form of a vector indicative of the proximal development of the 
evaluated person. Preferably, the results are in the form of a narrative, which is developed 
based on whether the prime question or oiie of the subcomponents was answered correctly 
and on which of the series of subcomponents was answered correctly. It will be apparent that 
the narrative may be developed from narrative components stored in a database. Such 
narrative con^onents may be sentence fragments fliat are assembled to form the narrative. 

Based on the foregbmg example, if the prime question **2 + 5 = ?" is answered 
correctly, the narrative migjit read, "Mr, Smifli understands the coiicept of adding two 
integers." If however, the prime question is answered incorrectly but the JBrst subcomponent 
is answered correctly, the narrative might read, "Mr: Smith understands the concept of adding 
two mtegers but appears to have some difficulty." Furthermore, if the prime question and 
first subcomponent are answered incorrecfly but the second subcomponent is answered 
correctly, the narrative might read, "Mr. Smith has difficulty adding two integers but seems to 
understand the general concq)t of addition." If the prime question and all of flie 
.5 corresponding subcomponents are answered incorrectly, die narrative might read, **Mr. Smidi 
appears to have difficulty with addition." 

Of course, the present invention is in no way limited to the foregoing exemplary 
embodiment. It will be apparent to those skilled in the art that the evaluation mefliod of the 
present invention may be appUed to any subject matter or combination tiiereot including, but 
in no way limited to mathematical subjects and the language arts. The evaluation metiiod of 
the present invention may also be applied to measure physical development or to measure 
visual or hearing abilities. The evaluation method may further be appUed to measure 
speaking abilities, vocabulary development, cognitive and reasoning skills, decision-making 
skills or the like. 

It will be appreciated that the foregoing method may be computer-controlled and that 
the method may be stored in a computer-readable medium as a set of instructions performed 
by a processor. The computer-readable medium may be, for example, a magnetic medium, 
optical medium, magneto-optical medium, CD-ROM, DVD-ROM, RAM or ROM or the like. 

Thus, die several aforementioned objects and advantages of die present invention are 
most effectively attained. Those skilled in th^ art will appreciate diat many modifications of 
die preferred embodunent described hereinabove may be made widiout departing from the 
sphit and scope of the invention. Alfliough a single preferred embodiment of the invention 
has been described and disclosed in detail heireii^ it should be understood that diis invention 



15 



10 



wo 02/01535 PCT/USOl/20416 

11 

is in no sense limited thereby and that its scope is to be detennined by that of the appended 
claims. 
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WHAT IS CLAIMED TS; 

1. A evaluation method for presenting a plurality of questions, the questions 
representing a matrix of a number of N items in each of a number of M tracks, the method 
comprising the steps of; 

(a) selecting a first track; 

G>) presenting the questions in item order from the first track selected in the selecting 
step (a) and receiving an answer in response to and corrfesponding to each question presented 
imtil a first condition is satisfied; 

(c) selecting another track when the first condition is satisfied; 

(d) presenting the questions in item order &om the second track selected in the 
selecting step (c) and receivmg an answer in response to and corresponding to each question 
presented until a second condition is satisfied; and 

(e) repeating the steps (c) arid (d) until a third condition is satisfied. 

2. The method according to claim 1, wherein the first condition is defined by 
receivmg an incorrect answer to one of the questions presented in the presmtmg step (b). 

3. The method according to claim 2, wherem the second condition is defined by 
receiving an incorrect answer to one of the questions presented in the presenting step (d). 

The method according to claim 1, wherein the third condition is defined by 
an incorrect answer to each of a sequence of questions presented in the presenting 

5. The method according to claim 1, further comprising the step of skipping a 
predetermined nimiber of questions in the selected track in response to a fourth condition. 

6. The method according to claim 5, wherein the fourth condition is defined by 
receiving a correct answer to each of a sequence of questions presented in one of the 
presenting steps (b) and (d). 
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7. The method according to claim 1, wherein the selecting step (c) includes the steps 

of: 

determining whether the track selected was previously selected in one of the selected 
steps (a) ?nd (c); and 

repeating the selecting step (c) imtil.the determining step determines that the track 
selected was not previously selected in one of the selecting steps (a) and (c). 

8. The method according to claim 1, wherein the third condition is defined by 
receiving a predetermined number of incorrect answers to the questions presented in the 
presenting step (d). 

9. The method according to claim 1, wherein the third condition is defined by 
receiving a predetermined number of incorrect answiers in sequence to the questions presented 
in the presenting step (d). 

10. The method according to claim 1, wherein the first track selected in the selecting 
step (a) is selected randomly* 

1 1 . The method according to claim 1 , wherein the another track selected in the 
selecting step (c) is selected randomly. 

12. The method according to claim 1, wherein the questions presented in the 
presenting step (b) are presented in order of increasing difficulty. 

13. The method according to claim 12, wherein the questions presented in the 
presenting step (d) are presented in order of increasing difficulty. 

14. The method according to claim 1, wherein selecting step (c) includes a step of 
selecting a new item level different firom the item level at which the second condition 
occuired 

15. The method according to claim 14, wherein the new item level represents a 
question of lesser difficulty than the item level at which the second condition occurred. 
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16. The method accordmg to claim 1, wherein each of the questions comprises a 
prime question and at least one subcomponent, the presenting and receiving step (b) 
comprising the substeps of: 

presenting the prime question; 

receiving an answer in response to the prime question; and 
presenting, each of the at least one subcon^onents if the answer received in 
response to the prime question presented is iDComci; and 

receiving an answer in response to each subcomponent presented; 
and the presenting and receiving step (d) comprising the substeps o£ 
presenting the prime question; 

' receiving an answer in response to the prime question; and 
presenting each of the at least one subcomponents if tiie answer received in 
response to flie prime question presented is incbrrect; and 

receiving an answer in response to each subcomponent presented. 

17. The mefliod according to claim 1, further comprising the step of developing a 
narrative result based on the questions presented and flie angers received. 

18. The method according to claim 17, wherein the developing step includes a step of 
assembling a series of narrative components stored in a database in accordance with tihie 
questions presented and die answers received. 

19. A computer-readable storage medium storing a set of instructions, the set of 
instmctions capable of being executed by a processor to implement an evaluation method for 
/iresenting a plurality of questions, the questions representing a matrix of a number of N 
items in each of a number of M tracks, the set of instructions performing the steps of: 

(a) selecting a first track; 

(b) presenting the questions in item order from the first track selected in the selecting 
step (a) and receiving an answer in response to and corresponding to each question presented 
imtil a first condition is satisfied; 

(c) selecting another track when the first condition is satisfied; 
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(d) presenting the questions in item order from the second track selected in the 
selecting step (c) and receiving an answer in response to and corresponding to each question 
presented until a second condition is satisfied; and 

(e) repeating the steps (c) and (d) until a third condition is satisfied. 

20. The Ciiinputer-readable medium according to claim 19, wherein the first condition 
is defined by receiving an incorrect answer to one of the questions presented in the presenting 
step(b). 

21. The computer-readable medium according to claim 20, wherein ttie second 
condition is defined by receiving an incorrect answer to one of the questions presented in tiie 
presentmg step (d). 

22. The computer-readable medium according to claim 19, wherein the third 
condition is defined by receiving an incorrect answer to each of a sequence of questions 
presented in the presenting step (d). 

23. The computer-readable medium according to claim 19, wherein the set of 
instructions fiirther performs the step of skipping a predeteimiined nuixiber of questions in the 
selected track in response to a fourth condition. 

. 24. The computer-readable medium according to claim 23, wherein the fourth 
condition is defined by receiving a correct answer to each of a sequence of questions 
presented in one of the presenting steps (b) and (d). 

25. The computer-readable meditmi according to claim 19, wherein the selecting step 
(c) includes the steps of: 

determining whether the track selected was previously selected in one of the selected 
steps (a) and (c); and 

repeating the selecting step (c) imtil the determining step determines that the track 
selected was not previously selected m one of the selecting steps (a) and (c). 
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26. The computer-readable meditim according to claim 19, wherein the third 
condition is defined by receiving a predetermined number of incorrect answers to the 
questions presented in the presenting step (d). 

27. The computer-readable medium according to claim 19, wherein the third 
condition is defined by receiving a predetermined number of incorrect answers in sequence to 
the questions presented in the presenting step (d). * 

28. The computer-readable medium according to claim 19, wherein the first track 
selected in the selecting step (a) is selected randomly: 

29. The con:i|>uter-readable medium according to claim 19, wherein the another track 
selected in the selecting step (c) is selected randomly. 

30. The computer-readable medium according to claim 19, wherein the questions 
presented in the presenting step (b) are presented in order of increasing difficulty. 

31. The computer-readable medium according to claim 30, wherein flie questions 
presented in the presenting step (d) are presented in order of increasing difiBculty. 

32. The computer-readable medixun accdrdiiig to claim 19, wherein selecting step (c) 
includes a step of selecting a new item level different fi:om the item level at which the second 
condition occurred. 

33. The computer-readable medixmi accorcl'ng to claim 32, wherein the new item 
level represents a question of lesser difficulty than the item level at which the second 
condition occurred. 

34. The computer-readable medium according to claim 19, wherein each of the 
questions comprises a prime question and at least one subcomponent, the presenting and 
receiving step (b) comprising the substeps of: 



presenting tiie prime question; 

receiving an answer in response to the prime question; and 
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presenting each of the at least one subcomponents if the answer received in 
response to the prime question presented is incorrect; and 

receiving an answer in response to each subcomponent presented; 
and the pr'^senting and receiving step (d) coinprising the substeps of: 
presenting the prime question; 

receiving an answer in response to the prime question; and 
presenting each of the at least one subcomponents if the answer received in 
response to the prime question presented is incorrect; and 

receiving an answer in response to each subcomponent presented. 

35. The computer-readable medium according to claim 19, wherein tiie method 
further comprises the step of developing a narrative result based on the questions presented 
and the answers received. 

36. The computer-readable medium according to claim 35, wherein the developing 
step includes a step of assembling a series of narrative con:q)onents stored in a database in 
accordance with the questions presented and the answers received. 

37. A con:q)uter system, comprising: 
a memory; 

an input device; 

an output device; and 

an evaluation method for presenting a plurality of questions, the questions 
representing a matrix of a number of N items in each of a number of M tracks, the method 
comprising the steps of: 



(a) selecting a first track; 



(b) presenting the questions in item order fi:om the first track selected in the 
selecting step (a) and receiving an answer in response to and corresponding to each 
question presented until a first condition is satisfied; 



(c) selecting another track when tiie first condition is satisfied; 



(d) presenting tiie questions in item order Scorn the second track selected in tiie 
selecting step (c) and receiving an answer in response to and corresponding to «ach • 
question presented until a second condition is satisfied; and 
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(e) repeating the steps (c) and (d) until a third condition is satisfied. 

38. The computer system according to claim 37, wherein the first condition is defined 
by receiving an incorrect answer to one of the questions presented in the presenting step (b). 

39. The computer system accordmg to claim 38, wherein the second condition is 
defined by receiving an incorrect answer to one of rhe quesddiis presented in the presenting 
step(d), 

40. The computer system according to claim 37, wherein the third condition is 
defined by receiving an incorrect answer to each of a sequence of questions presented in the ' 
presenting step (d). 

41. The computer system according to claim 37, wherein the method fijrflier 
conq)rises the step of skipping a predetermined rnunber of questions in the selected track in 
response to a fourth condition. 

42. The computer system according to claim 41, wherein the fourth condition is 
defined by receiving a correct answer to each of a sequence of questions presented in one of 
the presenting steps (b) and (d). 

43. The computer system according to claim 37, wherein the selecting step (c) 
includes the steps of: 

determining whether the track selected was previously selected in one of the selected 
rteps (a) and (c); and 

repeating the selecting step (c) until the determining step determines that the track 
selected was not previously selected in one of the selecting steps (a) and (c). 

44. The computer system according to claim 37, wherein the third condition is 
defined by receiving a predetermined number of incorrect answCTS to the questions presented 
in the presenting step (d). 
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45. The computer system according to claim 37, wherein the third condition is 
defined by receiving a predetermined number of incorrect answers in sequence to the 
questions presented in the presenting step (d). 

46. The computer system according to claim 37 wherein the first track selected in the 
selecting step (a) selected randomly. 

47. The computer system according to claim 37, wherein the another track selected in 
the selecting step (c) is selected randomly. 

48. The computer system according to claim 37, wherein Hxe questions presented in 
the presenting step (b) are presented in order of increasing difiBculty. 

49. The computer system according to claim 48, wherein the questions presented in 
the presenting st^ (d) are presented in order of increasing difiBculty. 

50. The coni^uter system according to claim 37, wherein selecting step (c) includes a 
step of selecting a new item level different firom the item level at which the second condition 
occurred. 

5 1 . The computer system according to claim 50, wherein the new item level 
represents a question of lesser difiBculty than the item level at which the second condition 
occurred. 

52. The computer system according to claim 37, wherein each of the questions 
comprises a prime question and at least one subcomponent, the presenting and receiving step 
(b) comprising the substeps of: 

presenting the prime question; 

receiving an answer m response to the prime question; and 
presenting each of the at least one siubcdn^onents if the answer received in 
response to the prime question presented is incorrect; and 

receiving an answer in response to each subcomponent presented; 
and Ihe presenting and receiving step (d) comprising the substeps of: 
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presenting the prime question; 

receiving an answer in response to the prime question; and 
presenting each of the at least one subcomponents if the answer received in 
response to the prime question presented is incorrect; and 

receiving an answer in response to each subcomponent presented. 

53. The computer system according to claim 37,' wherein the method further 
comprises the step of developing a narrative result based on the questions presented and the 
answers received. 

54. The computer system accordmg to claim 53, wherein the developing step includes 
a step of assembling a series of narrative components stored in a database in accordance with 
the questions presented and the answers received. 
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